Critical elements of synthesis questions are incompletely reported: survey of systematic reviews of intervention effects.

OBJECTIVES: To examine the characteristics of population, intervention and outcome groups and the extent to which they were completely reported for each synthesis in a sample of systematic reviews (SRs) of interventions. STUDY DESIGN AND SETTING: We coded groups that were intended (or used) for comparisons in 100 randomly sampled SRs of public health and health systems interventions published in 2018 from the Health Evidence and Health Systems Evidence databases. RESULTS: Authors commonly used population, intervention and outcome groups to structure comparisons, but these groups were often incompletely reported. For example, of 41 SRs that identified and/or used intervention groups for comparisons, 29 (71%) identified the groups in their methods description before reporting of the results (e.g., in the Background or Methods), 12 (29%) defined the groups in enough detail to replicate decisions about which included studies were eligible for each synthesis, 6 (15%) provided a rationale, and 24 (59%) stated that the groups would be used for comparisons. Sixteen (39%) SRs used intervention groups in their synthesis without any mention in the methods. Reporting for population, outcome and methodological groups was similarly incomplete. CONCLUSION: Complete reporting of the groups used for synthesis would improve transparency and replicability of reviews, and help ensure that the synthesis is not driven by what is reported in the included studies. Although concerted effort is needed to improve reporting, this should lead to more focused and useful reviews for decision-makers.

Effect estimates can be accurately calculated with data digitally extracted from interrupted time series graphs.

Interrupted time series (ITS) studies are frequently used to examine the impact of population-level interventions or exposures. Systematic reviews with meta-analyses including ITS designs may inform public health and policy decision-making. Re-analysis of ITS may be required for inclusion in meta-analysis. While publications of ITS rarely provide raw data for re-analysis, graphs are often included, from which time series data can be digitally extracted. However, the accuracy of effect estimates calculated from data digitally extracted from ITS graphs is currently unknown. Forty-three ITS with available datasets and time series graphs were included. Time series data from each graph was extracted by four researchers using digital data extraction software. Data extraction errors were analysed. Segmented linear regression models were fitted to the extracted and provided datasets, from which estimates of immediate level and slope change (and associated statistics) were calculated and compared across the datasets. Although there were some data extraction errors of time points, primarily due to complications in the original graphs, they did not translate into important differences in estimates of interruption effects (and associated statistics). Using digital data extraction to obtain data from ITS graphs should be considered in reviews including ITS. Including these studies in meta-analyses, even with slight inaccuracy, is likely to outweigh the loss of information from non-inclusion.

Synthesis methods other than meta-analysis were commonly used but seldom specified: survey of systematic reviews.

OBJECTIVES: To examine the specification and use of summary and statistical synthesis methods, focusing on synthesis methods other than meta-analysis. STUDY DESIGN AND SETTING: We coded the specification and use of summary and synthesis methods in 100 randomly sampled systematic reviews (SRs) of public health and health systems interventions published in 2018 from the Health Evidence and Health Systems Evidence databases. RESULTS: Sixty of the 100 SRs used other synthesis methods for some (27/100) or all syntheses (33/100). Of these, 54/60 used vote counting: three based on direction of effect, 36 on statistical significance, and 15 were unclear. Eight SRs summarized effect estimates (for example, using medians). Seventeen SRs used the term 'narrative synthesis' (or equivalent) without describing methods; in practice 15 of these used vote counting. 58/100 SRs used meta-analysis. In SRs providing a rationale for not proceeding with meta-analysis, the most common reason was due to diversity in study characteristics (33/39). CONCLUSION: Statistical synthesis methods other than meta-analysis are commonly used, but few SRs describe the methods. Improved description of methods is required to allow users to appropriately interpret findings, critique methods used and verify the results. Greater awareness of the serious limitations of vote counting based on statistical significance is required.

Effectiveness of social prescribing for chronic disease prevention in adults: a systematic review and meta-analysis of randomised controlled trials.

BACKGROUND: Social prescribing (SP) enables healthcare professionals to link patients with non-medical interventions available in the community to address underlying socioeconomic and behavioural determinants. We synthesised the evidence to understand the effectiveness of SP for chronic disease prevention. METHODS: A systematic literature search was conducted using five databases and two registries. Eligible studies included randomised controlled trials of SP among community-dwelling adults recruited from primary care or community setting, investigating any chronic disease risk factors defined by the WHO (behavioural factors: smoking, physical inactivity, unhealthy diet and excessive alcohol consumption; metabolic factors: raised blood pressure, overweight/obesity, hyperlipidaemia and hyperglycaemia). Random effect meta-analyses were performed at two time points: completion of intervention and follow-up after trial. RESULTS: We identified nine reports from eight trials totalling 4621 participants. All studies evaluated SP exercise interventions which were highly heterogeneous regarding the content, duration, frequency and length of follow-up. Majority of studies had some concerns for risk of bias. Meta-analysis revealed that SP likely increased physical activity (completion: mean difference (MD) 21 min/week, 95% CI 3 to 39, I2=0%; follow-up ≤12 months: MD 19 min/week, 95% CI 8 to 29, I2=0%). However, SP may not improve markers of adiposity, blood pressure, glucose and serum lipid. There were no eligible studies that primarily target unhealthy diet, smoking and excessive alcohol drinking behaviours. CONCLUSIONS: SP exercise interventions probably increased physical activity slightly; however, no benefits were observed for metabolic factors. Determining whether SP is effective in modifying the determinants of chronic diseases and promotes sustainable healthy behaviours is limited by the current evidence of quantification and uncertainty, warranting further rigorous studies. PROSPERO REGISTRATION NUMBER: CRD42022346687.

Strengthening systematic reviews in public health: guidance in the Cochrane Handbook for Systematic Reviews of Interventions, 2nd edition.

AIMS: Decision makers in public health practice and policy rely on access to trustworthy, relevant, synthesized evidence. The second edition of the Cochrane Handbook for Systematic Reviews of Interventions ('the Handbook') reflects a major revision in guidance for authors of systematic reviews, incorporating a decade of methodological development and a number of significant changes to previous recommendations. This paper aims to highlight new guidance that addresses a number of key methodological challenges for authors of systematic reviews in public health. RESULTS: The revised Handbook includes guidance on framing public health research questions for synthesis, considering equity, intervention complexity, risk of bias assessment and synthesis methods other than meta-analysis. Reviews of public health interventions frequently encounter the types of methodological complexity addressed in this new guidance. CONCLUSION: We hope that readers will find that the Cochrane Handbook includes detailed and thoughtful guidance on both conceptualizing and executing systematic reviews relevant to public health questions. Considering the available methods guidance will, we hope, provide support for authors of public health reviews to tackle the challenges they encounter, strengthen their analysis and provide useful answers to the important questions asked by stakeholders and users of public health evidence.

Understanding implementability in clinical trials: a pragmatic review and concept map.

BACKGROUND: The translation of evidence from clinical trials into practice is complex. One approach to facilitating this translation is to consider the 'implementability' of trials as they are designed and conducted. Implementability of trials refers to characteristics of the design, execution and reporting of a late-phase clinical trial that can influence the capacity for the evidence generated by that trial to be implemented. On behalf of the Australian Clinical Trials Alliance (ACTA), the national peak body representing networks of clinician researchers conducting investigator-initiated clinical trials, we conducted a pragmatic literature review to develop a concept map of implementability. METHODS: Documents were included in the review if they related to the design, conduct and reporting of late-phase clinical trials; described factors that increased or decreased the capacity of trials to be implemented; and were published after 2009 in English. Eligible documents included systematic reviews, guidance documents, tools or primary studies (if other designs were not available). With an expert reference group, we developed a preliminary concept map and conducted a snowballing search based on known relevant papers and websites of key organisations in May 2019. RESULTS: Sixty-five resources were included. A final map of 38 concepts was developed covering the domains of validity, relevance and usability across the design, conduct and reporting of a trial. The concepts drew on literature relating to implementation science, consumer engagement, pragmatic trials, reporting, research waste and other fields. No single resource addressed more than ten of the 38 concepts in the map. CONCLUSIONS: The concept map provides trialists with a tool to think through a range of areas in which practical action could enhance the implementability of their trials. Future work could validate the strength of the associations between the concepts identified and implementability of trials and investigate the effectiveness of steps to address each concept. ACTA will use this concept map to develop guidance for trialists in Australia. TRIAL REGISTRATION: This review did not include health-related outcomes and was therefore not eligible for registration in the PROSPERO register.

The use of 'PICO for synthesis' and methods for synthesis without meta-analysis: protocol for a survey of current practice in systematic reviews of health interventions.

Introduction: Systematic reviews are used to synthesise research and inform decision making by clinicians, consumers and policy makers. The synthesis component of systematic reviews is often narrowly considered as the use of statistical methods to combine the results of studies, primarily meta-analysis. However, synthesis can be considered more broadly as a process beginning with: (i) defining the groupings of populations, interventions and outcomes to be compared (the 'PICO for each synthesis'); (ii) examining the characteristics of the available studies; and (iii) applying synthesis methods from among multiple options. To date, there has been limited examination of approaches used in reviews to define and group PICO characteristics and synthesis methods other than meta-analysis. Objectives: To identify and describe current practice in systematic reviews in relation to structuring the PICO for each synthesis and methods for synthesis when meta-analysis is not used. Methods: We will randomly sample 100 systematic reviews of the effects of public health and health systems interventions published in 2018 and indexed in the Health Evidence and Health Systems Evidence databases. Two authors will independently screen studies for eligibility. One author will extract data on approaches to grouping and defining populations, interventions and outcomes, and the rationale for the chosen groups; and the presentation and synthesis methods used (e.g. tabulation, visual displays, statistical synthesis methods such as combining P values, vote counting based on direction of effect). A second author will undertake independent data extraction for a subsample of reviews. Descriptive statistics will be used to summarise the findings. Specifically, we will compare approaches to grouping in reviews that primarily use meta-analysis versus those that do not. Conclusion: This study will provide an understanding of current practice in two important aspects of the synthesis process, enabling future research to test the feasibility and impact of different methodological approaches.

Australian health policy makers' priorities for research synthesis: a survey.

OBJECTIVES: Health policy making is complex, but can be informed by evidence of what works, including systematic reviews. We aimed to inform the work of the Cochrane Effective Practice and Organisation of Care (EPOC) Group by identifying systematic review topics relevant to Australian health policy makers and exploring whether existing Cochrane reviews address these topics. METHODS: We interviewed 30 senior policy makers from State and Territory Government Departments of Health to identify topics considered important for systematic reviews within the scope of health services research, including professional, financial, organisational and regulatory interventions to improve professional practice and the organisation of services. We then looked for existing Cochrane reviews relevant to these topics. RESULTS: Eighty-five priority topics were identified by policy makers, including advanced practice roles, care for Indigenous Australians, care for chronic disease, coordinating across jurisdictions, admission avoidance, and eHealth. Sixty published Cochrane reviews address these issues, and 34 additional reviews are in progress. Thirty-four topics are yet to be addressed. CONCLUSIONS: This survey has identified questions for which Australian policy makers have indicated a need for systematic reviews. Further, it has confirmed that existing reviews do address issues of importance to policy makers, with the potential to inform policy processes.